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Introduction • • * ' . 

_ ' * 

OC Factor analysis has been the predominant technique used to interpret the 

LP structure of multiple-choice (M-C) achievement tests with binary response data. 

The major difficulty with Xhis approach lies in the meaningf\il interpretation 
r\J of the" resultant factor pattern matrix. Xhis difficulty stems from the fact 

r-H that the number of factors extracted often does not coincide wi^th the number 

CD* of factors predicted by theory, ^ ^ - 

LU ^Guilford (1941) and Horst (1966) suggested that the number, of factors 

extracted will be determined by the number of levels of difficulty. 'in the ^ 
test, even though the items themselves were homogeneous in respect to cQiltent 
McDonald and Ahlawat (1974) asserted that "spurious" factors - such as those o|;^i|| 
correlating with difficulty - are an artificat of the linear factor analytic '^oJ^^Sg 
> model. The implication of the McDonald-Ahlawat as^pttion is that there is noo^g^l^j 
. such thing as a "difficulty" factor; rather, that when impos^g a linear i>ll<>zl 
factor mqdel on curvilinear data, factors interpreted as/'spur^ous" or due ^^^IzSg^' 
."difficulty" are, in fact.^ fac-t^ors due to a curvilinear relation existing §fs<|^> 
between test scores and factor content. - »?>?5S9 

Gollob (1968a, 1968b) suggested that the item and person maio effects Jg^o^g|S 
are confounded with the persoji by item interaction. In attempting to developq^;^?|o ^ 
a modelj(»for inv^esti gating individual xdif f erences, he maintained th^t a "^jieg'S 
meaningful diBtinction exists between person , main effects and patteitns of ^^5??? 
"individual ^ff^erences as represented in the person by 't^st interaction. The' 
person main effect represents the individual's average performance across, tests 
in relation. to other individuals. The pattern of the' individual's pcrfcrmance, 
on the other hand, represehts how that individual performs on , any one test in 
relation to all other te^ts in the sample. In effect, this pattern of per- r 
formance is represented in the person by test interaction term in' the ANOVA 
model. Gollob 's model, the FANOVA model, i^ .essentially a 'factor analysis of 
.this interaction term. This is accomplished by subtraciii^ng out both'row 
(person) and column (test) main effects tq double center the data matrix,. then 
factoring the residual matrix (person by test interaction). , 

^Overallantl Klett (1972) have incorporated Gollob ' s logic . into a Linear 
Typal Analysis Program that is similar to profile analysis. ."'This program ||§||^ 
attempts to describe the dimensj^onality of the person by test interaction^ '^^^l! 
-component of the total variapj^'f rom a person, or Q, point of view, thus 
grouping the patterns of ifmividual differences according to. their similarity 1 1§ - o 
Ihis study hypothesized that the .patterns of individual differerices o^ll?' 
l^m^^ ^ resulting from student ' performance on aptitude tests will prove useful ip -^s^H^l 
^ «^ c.v.-;r.» ^ fnvr^nr.r>,v nf i^-^C ^^Prnc that will facilitate the interpvcLation 
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establishing a taxonomy of >t--C items that will faciMtate the interpi:cLation 
of the dimensionality of the M-C test, the connection between. the patterns 
* of individual differences and p-C items were established through the use of 

multiple discriminant function technique, where the M-C items were the dls- 
• xr\ crim"inator variables, and the groups of patterns of individual differences 

were the criteria. . , / H 
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Method 

The 112 subjects used in the' analysis were drawn from 256 first year 
students at Wayne State University School of Medicine. 

The M-C items were selected from an examination covering the intro- 
ductory unit compos^^ of biochemistry, gross anatomy and cell biology. A 
set of thirty-five .(35) items' were chosen according to two criteria; (1) a 
point bisereal coefficient between item scores and test scores of greater 
than .15 and, (2) an item difficulty of greater than .15 and less than .85. 
This set of thirty-five (35) items was used as discriminator variables in the 
discriminant function analysis. ^ 

Ihe aptitude measures were based on the- Structure of Intellect (SI) 
no3el of Guilford (1974). The SI tests used ^in this study were chosen on the 
basis qj[ an a priori analysis of the relationship between the items, on mul- 
tiple choice examinations from previous yeatrs and the a*bilities the items 
were purported to measure* As a result of this analysis, tests representing 
eighteen (18) cells^ of the Guilford cube were selected with two (2) tests 
per c^ll* , ^ • , 

The achievement tests were administered in the normal' course of the term. 
The aptitude tests'were administered to the entiri^ first year classes a 
group in one day, prior to beginning their academic studies. 

Results ' * ^ • 

The principal aice& method of factor analysis with iterations for 
communality estima^teS was used in an initial attempt to understand the 
dimensionality underlying the M-C test. The factors -were rotated to a > 
simple structure criterion using the direct oblimin* method of rotation, 
this^method could yield correlated factors. However, in all analyses, the 
correlations among factors was low. * < ^ 

The factor analysis of the thirty-five (35) items was not unequivocally 
interpretable by any frame of reference. Attempts were made "to interpret 
this fnatrpc using content, item difficulty, and' a revised Bloom taxonomy with 
no success » , ^ , ' ' 

Six of the SI tests were excluded from the analysis due to extreme 
skewnesg. The Liriear typal Analysis of the thirty (30) ^remaining tests^ 
indicated three clear groups of patterns of performance. The three (3). 
group's were identified as evaluation, cognition, anjd ''memory . 

The multiple discriminant function analysis resulted in a significant 
set of discriminant func\tions (F=l*70,. df=AO^ 180^ p^L.05) and Included 
twenty (20)* of, the thirty-five (35) items entered. It was quite apparent 
which items were jthe best discriminators between the grpups. 

'The factor pattern ml^trix of the thirty-five (35) M-C items was re- 
interpreted' on the b^sis qf the discriminant function analysis from the 
point of view of the relationship between that item and the group for which 
it best discriminated* Th^ is to say, if the it^ discriminated the 
cognition gpoup from the evaluation and memory groups, it was classified as a 
'Cognition questi''on* If an item discriminated the evaluation group from the 
cognition and memory groups, it was classified as an evaluation question* 
And, if an. item discriminated - the memory grouft from the evaluation and ^ 
cognition groups, it was considered a memory question. This re-interpretation, 
of' the factor pattern matrix, using the new taiconom^c scheme, made it quite 
apparent that the dimensions underlying "the multiple choice test were, to a 
large extent, representative of the type of ability called for to answer the 
question. " , * 



Discussion ' ' . 
1^ 

One of the maj-or findings of this study was £hat the interpretation of 
the dimensionality of an M-C examination from an individual differences point 
of view, was useful towards understanding that dimensionality. However, the 
fact' that this interpretation was not unequivocal stresses. the confounding 
of the item main ef-f ect with the person by item interaction. 

- The results reported in this study rai^e several questions- regarding 
the interpretation of M-C, examinations . First, since it has been shown that 
taxonomic systems can be further developed, to explai:^ the dimensionality of 
an examination, is it fruitful to continue'using, total scores with the con- 
comitant loss of information in assessing performance? - 

Second, it is readily apparent that persons differ in regard to ability, 
and M~C items are differentially related to these abilities, therefore, are 
achievement examinations being constructed that are biased against groups of 
students due to over-emphasis on the way in which items are constructed? 

Third, vl-ien there Is a, strong person by test interaction , that is^not' 
taken into account, might not the interpretation of the dimensionality of 
the M-C items be misleading and/or spurious? 
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